Comparison of Inter-Rater Reliability Techniques in Performance-Based Assessment
نویسندگان
چکیده
The aim of this study is to analyse the importance number raters and compare results obtained by techniques based on Classical Test Theory (CTT) Generalizability (G) Theory. Kappa Krippendorff alpha CTT were used determine inter-rater reliability. In descriptive research data consists twenty individual investigation performance reports prepared learners International Baccalaureate Diploma Programme (IBDP) also five who rated these reports. Raters an analytical rubric developed Organization (IBO) as a scoring tool. show that statistical failed provide information about sources errors causing incompatibility in criteria. studies G provided comprehensive increasing would increase reliability values. However, raised idea it important develop descriptors criteria rubric.
منابع مشابه
Comparison between inter-rater reliability and inter-rater agreement in performance assessment.
INTRODUCTION Over the years, performance assessment (PA) has been widely employed in medical education, Objective Structured Clinical Examination (OSCE) being an excellent example. Typically, performance assessment involves multiple raters, and therefore, consistency among the scores provided by the auditors is a precondition to ensure the accuracy of the assessment. Inter-rater agreement and i...
متن کاملValidity and Inter-rater Reliability Testing of Quality Assessment Instruments
ii This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted for which further reproduction is prohibited without the specific permission of copyright holders. The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No s...
متن کاملInter-rater reliability and Waterlow's pressure ulcer risk assessment tool.
AIM To ascertain whether a lack of inter-rater reliability with the original Waterlow (1996) pressure ulcer risk assessment scale is due to different perceptions of patients by nurses or different interpretations of Waterlow as a tool. METHOD A sample of 110 qualified nurses, who used the Waterlow pressure ulcer risk assessment scale in their daily work and were delegates at five study days, ...
متن کاملEvaluating Inter-rater Reliability of a National Assessment Model for Teacher Performance
This study addresses the high stakes nature of teacher performance assessments and consequential outcomes of passing versus failing based on decisions of those who subjectively score them. Specifically, this study examines the inter-rater reliability of an emerging national model, the Performance Assessment for California Teachers (PACT). Current reports on the inter-rater reliability of PACT u...
متن کاملInter-rater reliability of query/probe-based techniques for measuring situation awareness.
UNLABELLED Query- or probe-based situation awareness (SA) measures sometimes rely on process experts to evaluate operator actions and system states when used in representative settings. This introduces variability of human judgement into the measurements that require inter-rater reliability assessment. However, the literature neglects inter-rater reliability of query/probe-based SA measures. We...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Assessment Tools in Education
سال: 2022
ISSN: ['2148-7456']
DOI: https://doi.org/10.21449/ijate.993805